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Foreword 



A key mission of the Thomas B. Fordham Foundation is to supply education 
reformers with helpful, often actionable information that they might not 
otherwise see. In pursuit of that end, we sometimes ask experts — the kind 
whose writing is as clear as their analysis is careful — to distill significant 
research findings and explicate significant data. 

No education data are more significant than those that place U.S. schools and 
students in an international context and enable us to compare the performance 
of our education system with that of other countries. Although such analyses 
are resisted by some in the education fraternity — they typically claim that 
comparisons are inherently illegitimate and that America's situation is 
unique — most policymakers welcome them. And nobody we know is more adept 
at this sophisticated form of scholarship than Herbert J. Walberg, whose brief 
biography appears below. Tireless, meticulous, and seemingly immune to 
conventional wisdom, Dr. Walberg was a natural choice to examine and 
summarize important findings — drawn from several recent studies — that 
illuminate the academic performance of U.S. schools and students, and to 
incorporate in this analysis some helpful comparisons supplied by the 
Organization for Economic Cooperation and Development (OECD). 

The result is a vivid and unusual glimpse of the woeful productivity of U.S. 
schools in international perspective. 

The question Walberg set out to answer is: How much educational value do 
schools add as children pass through them, and at what cost? This approach 
eases the problems that typically dog comparative research, such as the fact 
that differing home environments and pre-school experiences mean youngsters 
in various countries start their formal schooling at different levels of readiness. 
When one looks at "value added," however, this factor practically disappears, 
and actual "school effects" are clearly revealed. 

The advanced countries that Walberg compared are all OECD members, found 
in North America, the Pacific Rim, and Western Europe. The OECD is a 
respected statistical and policy agency that obtains, analyzes, and reports 
uniform data on its member countries. Starting with economic statistics, the 
OECD added indicators and analyses of health, technology, and science. During 
the past decade, it launched an ambitious education indicator program 
embodied in its annual report, Education at a Glance: OECD Indicators. The 
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present report draws upon the latest three editions of that report, which include 
information on mathematics, reading, and science achievement. 1 

Dr. Walberg is Research Professor of Education and Psychology at the 
University of Illinois at Chicago. For a decade, he served on the Technical 
Committee of the International Association for the Evaluation of Educational 
Achievement (IEA), the multi-country agency that compares test scores of 
students in more than 25 countries. For the OECD, he chaired the technical 
advisory committee on education indicators. He also served as a founding 
member and chair of the Design and Analysis Committee of the National 
Assessment Governing Board. He is a member of the International Academy of 
Education and fellow of the American Association for the Advancement of 
Science, the American Psychological Association, the American Psychological 
Society, and the Royal Statistical Society (London). Readers wishing to contact 
him directly may write him at 180 East Pearson Street, Number 3607, Chicago, 
IL 60611-2107 or e-mail Hwalberg@uic.edu. 

The Thomas B. Fordham Foundation is a private foundation that supports 
research, publications, and action projects in elementary/secondary education 
reform at the national level and in the vicinity of Dayton, Ohio. Further 
information can be obtained from our web site (http://www.edexcellence.net) or 
by writing us at 1015 18 th Street, N.W., Suite 300, Washington, D.C. 20036. 

(We can also be e-mailed through our web site.) This report is available in full 
on the Foundation’s web site and hard copies can be obtained by calling 1-888- 
TBF-7474 (single copies are free). 

Chester E. Finn, Jr., President 

Thomas B. Fordham Foundation 

Washington, D.C. 

July 1998 
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Executive Summary 



Among schools in comparable countries, those in the U.S. on average make 
the smallest year-to-year gains in academic achievement. The longer 
American students are in school, the further they fall behind students in 
other lands. Yet per-pupil expenditures on U.S. schools are among the 
world's highest. 

Because they made the poorest progress and ranked in the highest category 
of spending, U.S. schools, by internationally agreed upon standards, are 
least productive among those in comparable (i.e., economically advanced) 
countries. 

More specifically: 

1) In reading, science, and mathematics through eighth grade, U.S. 
schools ranked last in four of five comparisons of achievement progress. 
In the fifth case, they ranked second to last. 

2) Between eighth grade and the final year of secondary school, U.S. 
schools slipped further behind those in other countries. 

3) Because they made the least progress, U.S. secondary schools ranked 
last in mathematics attainment and second to last in science. 

4) Per-student expenditures on U.S. primary and secondary schools were 
third highest among more than 20 advanced countries. 

5) Today, unlike the past, more secondary school students in comparable 
countries on average remain in school than in the U.S. Thus, their 
superior achievement gains do not depend on student selectivity or 
higher dropout rates. 

This report is based on the largest, most recent, and most rigorous 
international achievement surveys. These were carried out by the multi- 
national International Association for the Evaluation of Educational 
Achievement (IEA) and used by the Paris-based Organization for Economic 
Cooperation and Development (OECD) in its compilations of education 
indicators for economically advanced countries. This report makes use of 
the results for all age groups and employs measures of value-added 
progress made by schools rather than comparisons of students of a single 
age. 
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Spending More While 
Learning Less 

Reading Progress 

No skill is more fundamental to academic success than reading. Yet it is strongly 
influenced by socio-economic status, child rearing, and suchlike. Until children start 
school around the age of six, families, mass media, and other non-school factors are 

the chief sources of influence on their initial vocabulary 
and comprehension. Comparisons of schools that do not 
account for these factors would be unfair and 
incomplete. The big question about schools, then, is not 
how much students know at one point in time, especially early in their academic 
careers, but how much progress students make during the school years. In the case of 
reading, one recent measure of progress is available: gains made by students from 
ages 9 to 14. 2 

As shown in Chart 1, among OECD countries, U.S. students made the least progress 
in reading. 3 The average progress (difference between 9- and 14-year-olds) for the 16 
participating countries was 160 points. 4 U.S. students' progress was only 125 points, 
or 78 percent of the international average. 5 

Mathematics Progress 

Perhaps the most comparable of all subjects, mathematics has long been a topic of 
international achievement surveys. U.S. student attainments have generally ranked 
poorly, especially in the upper grades. What about the value added by schools? 



Among OECD countries , 
U.S. students made the 
least progress in reading . 
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Chart 1 Reading Progress 
(Ages 9 to 14) 




Note: The scale displayed here is a statistical construct, designed to illuminate the relative magnitude of achievement gain differences 
between age cohorts and among countries. These "points" cannot be calibrated directly to test scores or other "real-world" gauges. To 
grasp the order of magnitude shown in these achievement gains, it is helpful to note that in this case Denmark gained the most points, 
209; the U.S. the least, 125. 
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In the 1996 and 1997 Education at a Glance editions, the OECD reported analyses of 
data from the Third International Mathematics and Science Study. As shown in 
Chart 2, U.S. students made the least progress between grades seven and eight 
among 24 countries. 6 The average gain of all countries was 33. U.S. students gained 
only 24 points, or 73 percent of the international average. 

U.S. students made the Chart 3 shows fourth to eighth grade math gains of the 

least progress between 17 participating OECD-selected countries. U.S. schools 

grades seven and eight ran ^ i as t ; gaining 93 points, which is 73 percent of the 

among 24 countries. 

127 points gained by the average country. 

The latest report of the Third International Mathematics and Science Study (TIMSS) 
presents math and science data for eighth grade and the final year of secondary 
school for 16 OECD countries. Since different and uncalibrated tests were employed 
at the two grade levels, no direct comparison can be made between the performance of 
the upper secondary school students and that of eighth-graders. 8 This means that 
gains cannot be calculated without elaborate and possibly controversial test equating. 
The TIMSS team could, however, calculate the country averages and display their 
rank order as shown in Table I. From these data, the changes in rank order can 
readily be calculated and are shown in the last column of Table I. 

Am ong the 16 countries, those in Northern Europe gained the most ranks, while 
those in Eastern Europe lost the most. U.S. schools declined from fourteenth to last 
place. As shown in Table I, they dropped two ranks: from third last to last place. They 
could, of course, drop no further (unless the nonrOECD participating countries of 
Cyprus, Lithuania, and South Africa were included). 9 
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Chart 2 Mathematics Progress 
Seventh to Eighth Grade 

France 
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Germany 
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Note: The scale displayed here is a statistical construct, designed to illuminate the relative magnitude of achievement gain 
differences between age cohorts and among countries. These "points" cannot be calibrated directly to test scores or other 
"real-world" gauges. To grasp the order of magnitude shown in these achievement gains, it is helpful to note that in this case 
France gained the most points, 46; the U.S., the least, 24. 
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Chart 3 Mathematics Progress 
Fourth to Eighth Grade 
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Note: The scale displayed here is a statistical construct, designed to illuminate the relative magnitude of achievement 
gain differences between age cohorts and among countries. These "points" cannot be calibrated directly to test scores or 
other "real-world" gauges. To grasp the order of magnitude shown in these achievement gains, it is helpful to note that in 
this case Iceland gained the most points, 149; the U.S., the least, 93. 
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Table I: Changes in Rank in Mathematics Among 16 Countries 



Country 


Eighth Grade 


Final Year 


Change in Rank 


Iceland 


16 


5 


+11 


Denmark 


13 


3 


+10 


Norway 


15 


6 


+9 


Sweden 


10 


2 


+8 


New Zealand 


12 


8 


+4 


Netherlands 


3 


1 


+2 


Canada 


9 


9 


0 


Germany 


11 


12 


-1 


Switzerland 


2 


4 


-2 


U.S.A. 


14 


16 


-2 


France 


5 


7 


-2 


Australia 


8 


11 


-3 


Austria 


4 


10 


-6 


Hungary 


6 


14 


-8 


Russia 


7 


15 


-8 


Czech Republic 


1 


13 


-12 



Science Progress 

Twenty-four OECD countries participated in the seventh- to eighth-grade science 
portion of TIMSS. Shown in Chart 4, U.S. schools ranked last in value-added gains 
from seventh to eighth grade. The average country's schools gained 41 points; U.S. 
schools gained 26 points or 63 percent of the international average. 



Chart 5 shows the science gains of 17 countries from fourth to eighth grade. U.S. 
schools ranked second to last. They gained 113 points or 78 percent of the 144 point 
average country-gain. 



From, eighth to twelfth 

grade , U.S. schools declined A® in the mathematics part of TIMSS, the tests for 
from eighth to fifteenth the eighth grade and final year of secondary 

place , which means second school were not calibrated with one another. 

from the bottom. 

Therefore, TIMSS published the achievement 

ranks rather than the value-added gains for the two groups in the 16 participating 
countries. These and the changes in ranks are shown in Table II. 



Chart 4 Science Progress 
Seventh to Eighth Grade 
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Note: The scale displayed here is a statistical construct, designed to illuminate the relative magnitude of achievement 
gain differences between age cohorts and among countries. These "points cannot be calibrated directly to test scores or 
other "real-world" gauges. To grasp the order of magnitude shown in these achievement gains, it is helpful to note that in 
this case Russia gained the most points, 54; the U.S., the least, 26. 
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Chart 5 Science Progress 
Fourth to Eighth Grade 
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Note: The scale displayed here is a statistical construct, designed to illuminate the relative magnitude of 
achievement gain differences between age cohorts and among countries. These "points" cannot be calibrated directly 
to test scores or other "real-world" gauges. To grasp the order of magnitude shown in these achievement gains, it is 
helpful to note that in' this case Hungary gained the most points, 175; Korea, the least, 105; the U.S., second least 
113. 
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Again, the schools in Northern Europe generally attained the most by the final year 
and gained the most ranks. These countries generally also tested the largest 
proportions of students. Eastern European and U.S. schools not only attained the 
least but also lost the most ranks. U.S. schools declined from eighth to fifteenth place, 
which means second from the bottom. Since U.S. schools tested a smaller percentage 
of the school-leaving age cohort than most countries, its scores are probably 
overestimated. 

Table II: Changes in Rank in Science Among 16 Countries 



Country 


Eighth Grade 


Final Year 


Change in Rank 


Iceland 


15 


3 


12 


Norway 


11 


4 


7 


Sweden 


7 


1 


6 


New Zealand 


12 


6 


6 


Switzerland 


13 


7 


6 


Denmark 


16 


10 
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Canada 


10 


5 
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France 


14 


12 
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Netherlands 
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Germany 
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11 


-2 


Australia 
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-4 


Austria 


3 


8 


-5 


U.S.A. 


8 


15 


-7 


Russia 


6 


14 


-8 


Czech Republic 
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13 


-12 


Hungary 


4 


16 


-12 
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School Expenditures 

An ideal analysis of value added would measure both benefits and expenditures in 
monetary terms. Unfortunately, analysts cannot yet measure the dollar value of a 
point or rank gain on academic achievement tests. Nonetheless, for a more complete 
picture of value added, expenditures on schools can be reviewed in relation to 
achievement gains. The figures reported here are the OECD-calculated per-student 
expenditures for primary/secondary (public and private) schools in 1994. 10 These 1994 

ZTZi 7 w , , 7~. w expenditures are the latest available and 

U.S. schools ranked third 

highest among 22 countries in correspond to the y ears for which the 

per-student expenditures on achievement scores were obtained. 

primary schools and third 

highest among 23 countries .,. TTO , , 

, . _ Chart 6 shows that U.S. schools ranked third 

on secondary schools. 

highest among 22 countries in per-student 

expenditures on primary schools. At $5300, U.S. primary schools expended 75 percent 
more than the $3033 international average. Chart 7 shows that U.S. schools also 
ranked third on secondary school expenditures among 23 countries. At $6680, U.S. 
primary schools spent 54 percent more than the $4335 average of all countries. 
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Chart 6 Per-Student Expenditures on 
Primary Schools in Dollars 
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Chart 7 Per-Student Expenditures on 
Secondary Schools in Dollars 
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Conclusions 



The largest and most rigorous international achievement surveys ever conducted 
serve as the basis of the foregoing conclusions. For the first time, this report arrays in 
one place the rankings of the progress of economically advanced countries for all ages 
and school subjects recently surveyed. 

The rankings show that in the subjects surveyed — reading, science, and 
mathematics — U.S. schools ranked last in four of five comparisons of achievement 
progress. In the fifth case, they ranked second to last. Between eighth grade and the 

U.S. schools can 

termed the least behind other countries. Because they made generally 

productive among those the poorest progress, at the end of secondary school 

in economically U.S. students ranked last in mathematics and second to 

advanced countries. 

last in science among those in advanced industrial 

countries. Nonetheless, U.S. expenditures on primary and secondary schools were 
third highest among more than 20 countries. 

In short, U.S. schools are near the top in spending but last or near last in 
achievement progress. For these reasons, as measured by these internationally 
agreed upon standards, U.S. schools can fairly be termed the least productive among 
those in economically advanced countries. 



fairly he 



final year of high school, the U.S. slipped further 
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Appendix 



Value-Added Comparisons 

International achievement comparisons reveal important facts about students and 
schools. Policymakers especially want to know how students at the end of secondary 
school compare since it reveals their preparedness for higher education and the 
workforce. For this reason, nearly all previous reports have focused on comparisons of 
students of the same age or with the same number of years of schooling. 

In recent years, however, policymakers in the U.S. and elsewhere have become more 
concerned about the effectiveness of schools in inducing greater levels of learning. 
Static comparisons of primary schools — the kind typically made in the past — are less 
useful for this purpose because students' scores are partly determined by their 
experiences before they begin school, experiences heavily influenced by socioeconomic 
status and other external factors. Thus, gains in achievement during the school years 
are better gauges of schools' contributions to learning than are scores at a single point 
in time. 

Gains, progress, and value added — terms used synonymously here — are particularly 
important for policy because they allow predictions of eventual attainments. Policies 
that do not add sufficient value may be revised. Units of the system such as primary 
and secondary schools may be separately evaluated by measuring students' progress 
while under their jurisdiction. In addition, many experts (and laymen) believe that 
incentives influence performance. For this reason, managers give merit raises for 
recent progress rather than work-life attainments. If carrots and sticks were 
employed in education, value added rather than cumulative attainment would be the 
most reasonable indicator. 
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Education policymakers increasingly recognize the usefulness of value-added 
indicators. Internationally, the OECD pioneered the use of value-added indicators in 
the 1995 edition of Education at a Glance and has employed them in subsequent 
reports. Similarly, Dallas, Texas and the state of Tennessee are employing value- 
added indicators and incentives to increase school productivity. 11 Other cities and 
states such as Chicago and Virginia employ attainment indicators to assign schools to 
probation. Such systems tend to include schools that serve poor children but are not 
necessarily ineffective as schools. A fairer and more efficient evaluation system would 
employ value-added indicators as at least one consideration in evaluating schools. 

Reports sponsored by the U.S. National Academy of Sciences and the Brookings 
Institution recommend value-added indicators and incentives to boost the 
productivity of American schools. 12 A United Kingdom report to the Secretary of State 
for Education advises "that it is possible and desirable to set up a national system to 
provide schools with value-added indicators of their performance." According to the 
report, a national value-added system would be cost effective, readily understandable, 
and statistically valid. Value-added indictors can be easily calculated from 
standardized test scores and do not require changes in tests or testing procedures. 13 

For all these reasons, the present report compiles the value-added school 
achievement indicators in the most recent three (1995-1997) editions of the OECD's 
Education at a Glance. This report also includes a subsequent survey of secondary 
school students by the staff of the Third International Mathematics and Science 
Study. These information sources include indicators of reading, mathematics, and 
science achievement progress during the primary and secondary school years. 14 
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An Example 



Value-Added Reading Programs 

Reading may be the most fundamental skill for acquiring knowledge in schools. 
Reading vocabulary and comprehension scores predict not only listening, writing, and 
speaking skills, but also achievement in mathematics, science, social studies, foreign 
languages, and other school subjects. Though policymakers and scholars emphasize 
math and science in international comparisons, reading seems at least as important. 

Reading proficiency, moreover, is also of huge economic and social significance. 
Bormuth s 1970 s survey of about five-thousand people aged 16 and over showed 
that 87 percent of those employed reported that they had to read as part of their jobs. 
Typical working people read for 141 minutes per day as part of their jobs, or about 29 
percent of the workday. Since the national wage bill in 1971 was $859 billion, 
Bormuth estimated that U.S. workers earned $253 billion for on-the-job reading. 
Because there are more workers today, because they undoubtedly read even more, 
and because their hourly wages have increased, the amount paid for on-the-job 
reading is substantially greater today. Arguably, U.S. citizens are paid more for 
reading than for any other single activity. 

Bormuth's survey also showed that much leisure time is spent reading. Seventy-three 
percent of the sample, for example, reported having read a newspaper the previous 
day (similar to recent surveys); the mean reading time for this activity was 33 
minutes. It appears that average adults spend at least three hours a day reading. 



Perhaps better than any other subject or skill, reading also illustrates the need for 
value-added indicators when evaluating schools. National comparisons of attainment 
scores for a given age group could be misleading measures of school effects. Countries 
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vary substantially in determinants of reading skills, for example, due to differences in 
preschool and child-care policies, family practices, the extent of bilingualism, age of 
starting school, family income, and cultural cohesion. Since countries vary in these 
respects, the OECD devised an index of progress during the school years as a way of 
producing valid cross-national comparisons. 

In its 1995 report, Education at a Glance, the OECD raised the following policy 
questions: How much does reading literacy improve between the ages of 9 and 14? If 
progress is insufficient, what factors are at work? Is education policy one of them? So 
that national policymakers could answer such questions for themselves, the OECD 
devised the indicator, "progress in reading achievement." It is the difference in the 
reading achievement of 9- and 14-year-old students measured in nationally 
representative random samples obtained by the IEA between October 1990 and April 
1991. 16 

This value-added indicator is well suited for measuring school progress. By age 9, 
children in all OECD countries have several years of primary school under their 
belts. By age 14, almost none have dropped out of secondary school. Differences 
between age groups' scores remove early differences in readiness, socioeconomic 
status, and other conditions as much as education research allows. 



Notes 



1. Education at a Glance: OECD Indicators. (Paris, France: Organization for Economic 
Cooperation and Development, 1995, 1996, and 1997). 

2. As has been widely reported, U.S. 9-year-olds scored well in reading. In fact, they were 
second only to Finland's students. 

3. Strictly speaking, a few countries, such as Korea, were candidates rather than 
members of the OECD during the periods the reports were being prepared and 
published. They were included in the OECD reports and in the present report. 

4. As in the case of the Scholastic Aptitude Test and other common tests, the points on 
the scale referred to here — and displayed in Chart 1 — are a statistical construct, 
designed to illuminate the magnitude of differences between age cohorts and among 
countries. They cannot be calibrated to the percentage or number of points correct on 
tests with a fixed number of items. To convey the relative magnitude of achievement 
gains, a note beneath the chart states the greatest gain, the least gain, and the U.S. 
gain. 

5. It might be thought that the reading test was insufficiently challenging for U.S. 
students to show gains. Five of the 16 countries, however, had higher scores for 14- 
year-olds than the U.S. 

6. For seventh to eighth grade gains in mathematics and science, see Education at a 
Glance, 1996, pp. 212-213. The average standard error for the means is about five. 

7. For fourth to eighth grade gains in mathematics and science, see Education at a 
Glance, 1997, pp. 306-307. The average standard errors are 9 for mathematics and 14 
for science. Eight countries (including the U.S.) did not meet or partially met the 
sampling standards. The U.S. success at meeting inclusion standards at fourth grade 
and not at eighth grade would tend to lead to overestimates of U.S. gains. 

8. Ina V. S. Mullis, Michael O. Martin, Albert E. Beaton, Eugenio J. Gonzalez, Dana L. 
Kelly, and Teresa A. Smith, Mathematics and Science Achievement in the Final Year 
of Secondary School. (Chestnut Hill, Mass.: Boston College, TIMSS International 
Study Center, February 1998, pp. 53-54). 

9. U.S. schools' poor attainment and drop in ranks cannot be attributed to testing less 
elite students. TIMSS researchers calculated the proportion of the entire age cohort 
tested by each country. Twelve of the 16 countries tested a greater proportion than the 
U.S. The country average tested was 70 percent; the U.S. percentage was 63 percent. 
So, the U.S. estimates are probably biased upward more than the average of all other 
countries. Unlike the past, moreover, other comparable OECD countries now retain on 
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average higher percentages of 17- and 20-year-olds in school and postsecondary 
education (OECD, Education Policy Analysis. Paris: OECD, 1997, p. 98). 

10. Education at a Glance, 1997, pp. 92-101. The OECD adjusted the expenditure 
estimates for the purchasing power of each country's currency. 

11. See, for example, Jason Millman (Ed.), Grading Teachers, Grading Schools. 

(Thousand Oaks, Calif.: Corwin Press, 1997). 

12. Eric A. Hanushek and Dale W. Jorgenson (Eds.), Improving America's Schools: The 
Role of Incentives . (Washington, DC: National Research Council, 1966) and Brookings 
Institution panel of 13 economists, Making Schools Work: Improving Performance and 
Controlling Costs. (Washington, DC: 1994). 

13. Qualifications and Curriculum Authority, The Value Added National Project: Report to 
the Secretary of State. (London, Eng.: June 1997). The Authority recommended the 
simple difference between scores of older and younger students rather than more 
elaborate calculations recommended by some authorities. Simple gains are more 
readily understandable and produce very similar results as elaborate, time 
consuming, and debatable formulas. Advised by international authorities, the OECD 
employs simple differences. 

14. As the term is used internationally, the term primary schools refers to what many 
Americans call elementary or grade schools. Secondary schools refers to what 
Americans call high schools. 

15. John R. Bormuth, “Value and Volume of Literacy.” Visible Language, 1978, 12, 118-61. 

16. See footnote 1 , Education at a Glance, 1995, pp. 207-208. Called "synthetic cohort 
gains," these indicators are not gains of the same students at two time points but the 
(cross-sectional) differences between younger and older students at a single time 
point. For technical details about the reading test, see the OECD report and Warwick 
B. Elley (1992), How in the World do Students Read 1 ? (Amsterdam, The Netherlands: 
The International Association for the Evaluation of Educational Achievement, 1992) 
and the OECD reports cited in forgoing sections. 

Because the standard errors in the reading chart average about five points, countries 
whose average scores are close to one another may not differ significantly. The same 
principle applies to other comparisons in this report. It suggests that, if the surveys 
were repeated, countries that scored similarly in one survey could change rank order 
with one another. With variations in subjects and ages, the progress surveys were in 
fact repeated five times. U.S. schools came in last in four instances and second last in 
one, which is highly unlikely to have occurred by chance. 
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